HIERARCHICAL REINFORCEMENT LEARNING WITH FUNCTION APPROXIMATION FOR ADAPTIVE CONTROL by MARGARET

by MARGARET MARY SKELLY This dissertation investigates the incorporation of function approximation and hierarchy into reinforcement learning for use in an adaptive control setting through empirical studies. Reinforcement learning is an artificial intelligence technique whereby an agent discovers which actions lead to optimal task performance through interaction with its environment. Although reinforcement learning is usually employed to find optimal problem solutions in unchanging environments, a reinforcement learning agent can be modified to continually explore and adapt in a dynamic environment, carrying out a form of direct adaptive control. In the adaptive control setting, the reinforcement learning agent must be able to learn and adapt quickly enough to compensate for the dynamics of the environment. Since reinforcement learning is known to converge slowly to optimality in stationary environments, the use of abstraction and changes in task representation are examined as a means to accelerate reinforcement learning. Various levels of abstraction